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ABSTRACT 


During the first year of this project, a taxonomy of theoretical levels of automation (LOAs) was 
applied to the advanced commercial aircraft by categorizing actual modes of McDonald Douglas 
MD-11 autoflight system operation in terms of the taxonomy. As well, high LOAs included in 
the taxonomy (e.g., supervisory control) were modeled in the context of MD-11 auto flight 
systems through development of a virtual flight simulator. The flight simulator was an 
integration of a re-configurable simulator developed by the Georgia Institute Technology and 
new software prototypes of auto flight system modules found in the MD-11 cockpit. In addition 
to this work, a version of the Situation Awareness Global Assessment Technique (SAGAT) was 
developed for application to commercial piloting tasks. A software package was developed to 
deliver the SAGAT and was integrated with the virtual flight simulator. 


SITUATION AWARENESS & LEVELS OF AUTOMATION 
Summary of Project Achievements: 

During the first project year, the theoretical taxonomy of levels of automation (LOAs) developed 
by the PI and subcontractor were applied to the MD-11 autoflight systems. Specifically, existing 
modes of MD-1 1 automation were categorized in terms of the LOA taxonomy and new modes of 
auto flight not currently available in the aircraft were conceptualized based on function allocation 
schemes characterizing higher levels of complex system autonomy identified in the taxonomy. 
This was accomplished by assuming the utilization of an expert system in the Flight 
Management System (FMS) of the aircraft for flight path planning. 

General direction and outcomes: 

This effort was intended to describe how pilots and flight management and control computers are 
integrated in the advanced commercial aircraft cockpit. It also served as the basis for developing 
a virtual flight simulator of the MD-11. The simulator was an integration of a re-configurable 
flight simulator developed by the Georgia Institute Technology and new software prototypes of 
autoflight system modules found in the MD- 1 1 cockpit. The virtual flight simulator was 
developed on a personal computer platform running the Microsoft WindowsNT® operating 
system. 

The study of the MD-11 cockpit automation and pilot flight tasks also allowed for the 
development of a version of the Situation Awareness Global Assessment Technique (SAG AT) 
for commercial piloting. A software package was developed to deliver the SAGAT and it has 
been integrated with the virtual flight simulator. 

Specific tasks: 

Towards applying the LOA taxonomy to the MD-1 1 cockpit, we conducted an in-depth review 
of cockpit systems and flight tasks. This included: 

(1) learning the Delta Pilot’s Reference Manual; 

(2) visiting the Delta training facilities in Atlanta, Georgia and receiving familiarization with 
Delta MD-11 simulators; 

(3) consulting with experienced MD-1 1 pilots; and 

(4) developing hierarchical breakdowns of MD-1 1 autoflight system functions. 

The hierarchies established what the autoflight systems are capable of doing. They include a list 
of available functions and functional settings of the aircraft control systems. 

Subsequently, formal descriptions of the division of flight control responsibilities among pilots 
and flight management and control computers were developed for each mode of automation the 
MD-1 1 is currently capable of achieving (e.g., Vertical Speed Mode, FMS Navigation Mode, 
Autopilot Mode, Flight Director Mode, Autothrottle System, etc.). Descriptions of the channels 
of communication between pilots and on-board computers were also developed along with 
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descriptions of triggers of different modes of automation (i.e., flight events that would cause 
certain modes to be engaged or disengaged). 


A two-step procedure was used to categorize the modes of automation of the MD-1 1 in terms of 
the taxonomy LOAs. Initially, all flight control functions were classified according to the general 
stages of information processing represented in the taxonomy (i.e., monitoring, planning, 
decision-making and control action). Flight control function ownership under the various modes 
of MD- 1 1 automation was then considered and matched to the generic function allocation 
schemes depicted by the taxonomy LOAs. This served to identify the LOAs of the taxonomy 
currently represented in the MD-1 1 auto flight system. It also identified the levels that needed to 
be conceptually modeled as part of the flight simulator development in order to study their 
potential effects on flight performance and pilot situation awareness (SA). As a result of this 
process, it was observed that the modes of MD-1 1 automation, in general, constitute low degrees 
of aircraft autonomy and often only provide pilots with assistance in implementing flight control 
actions. 

In order to model higher LOAs in the context of the MD- 1 1 cockpit, such as supervisory control 
or full automation of aircraft flight control, the integration of an expert system in the Flight 
Management Computer (FMC) was assumed for flight path planning tasks. The system was 
conceptualized as being capable of: 

(1) automatically generating flight plans; 

(2) evaluating the safety and efficiency of flight plans, including plans generated by pilots; and 

(3) deciding on an optimal flight plan among several alternatives. 

By considering the aspects of high LOAs as part of the theoretical taxonomy, we identified the 
existing MD-11 auto flight systems that would need to be active in conjunction with the expert 
system to facilitate a particular LOA (e.g., supervisory control, etc.). We also defined who (the 
pilot or FMC) was to be responsible for specific flight tasks and control systems based on the 
generic function allocation schemes presented in the taxonomy. 

Deliverables: 

Description of Endsley & Kaber’s (1999) taxonomy of LOAs in terms of MD-11 autoflight 
systems and the operational roles of the pilot and flight management and control computers was 
submitted to the technical contact as part of a technical report entitled, “Situation Awareness & 
Levels of Automation,” on March 6, 2000. The report also presented comprehensive reviews of 
the MD-1 1 Pilot’s Flight Manuals, including “Aircraft General,” “Automatic Flight System,” and 
“Flight Control.” These reviews are important because they are essentially translations of pilot 
references, which use cryptic terminology, into laymen’s terms. It is possible that the reviews 
could be used by McDonald Douglas Corporation, Boeing Corporation and other commercial 
aircraft manufacturers to determine what information is actually being conveyed to new MD-11 
pilots through the autoflight system manuals. The reviews could also serve as a guide to 
McDonald Douglas Corporation safety for improving aircraft manuals to prevent piloting errors 
due to a lack of adequate system understanding. 

Beyond these reviews, the technical report also included the hierarchical analyses of the 
functions of the various MD-11 autoflight system functions (i.e., mode trees of automatic flight 
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system functions). These mode trees very clearly define the capabilities of the current modes of 
automation of the aircraft. They also provide insight into how pilots and the FMS and the FMC 
communicate during autoflight and flight events that may causes changes in the mode of aircraft 
automation. 

Other Research Outcomes: 

At this point in time, the project has generated four publications, including a refereed conference 
proceedings paper, an archival journal article and two software applications with commercial 
market potential. Current references for these publications include: 

(1) Endsley, M. R. (2000). Situation awareness global assessment technique for commercial 

piloting ©. Atlanta, GA: SA Technologies Inc.. 

(2) Kaber, D. B., Tan, K-W., Riley, J. and Endsley, M. (in review). Design of adaptive 

automation for complex systems. Submitted to the International Jo urnal of Cognitive 
Ergonomics . 

(3) Tan, K-W, Butts, D. and Song, D. (2000). Virtual MD-11 flight simulator ©. Mississippi 

State, MS: Mississippi State University. 

(4) Tan, K-W., Kaber, D. B., Riley, J. M. and Endsley, M. R. (in press). Human factors issues in 

implementation of AA to complex systems. In the Proceedings of the 14 th Triennial 
Congress of the International Ergonomics Association and the 44 th Annual Meeting o f the 
Human Factors & Ergo no mics Society . Human Factors & Ergonomics Society: Santa 
Monica, CA. 

The proceedings paper is to be presented at the “International Ergonomics Association/Human 
Factors & Ergonomics Society 2000 Meeting” this Summer in San Diego, California. It is 
anticipated that the journal article will be published during the winter of 2000. It is also expected 
that revisions will be made to both software packages during the second project year to enhance 
the simulator capabilities for recording data on pilot performance and to expand the query base 
of the S AGAT application. 

References: 


Endsley, M. R. and Kaber, D. B. (1999). Level of automation effects on performance, situation 
awareness and workload in a dynamic control task. Ergonomics, 42(3), 462-492. 


3 


Personnel: 


Biographical Sketch: Mica R. Endslev, Ph.D., P.E., CPE 

Mica Endsley is President of SA Technologies in Marietta, Georgia where she specializes in 
situation awareness issues in advanced aviation systems. She recently left her post as a Visiting 
Associate Professor at MIT in the Department of Aeronautics and Astronautics and Associate 
Professor of Industrial Engineering at Texas Tech University. Prior to joining Texas Tech she 
was an Engineering Specialist for the Northrop Corporation, serving as Principal Investigator of 
a research and development program focused on the areas of situation awareness, mental 
workload, expert systems and interface design for the next generation of fighter cockpits. She 
received a Ph.D. in Industrial and Systems Engineering from the University of Southern 
California. Dr. Endsley has been conducting research on situation awareness, decision making 
and automation in aircraft, air traffic control and aviation maintenance for the past twelve years. 
She is the author of over 90 scientific articles and reports on numerous subjects including the 
implementation of technological change, the impact of automation, the design of expert system 
interfaces, new methods for knowledge elicitation for AI system development, pilot decision 
making, and situation awareness. 
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Biographical Sketch: David B. Kaber. Ph.D., E.I.T. 


David Kaber is an Assistant Professor in the Department of Industrial Engineering at Mississippi 
State University and Director of the Cognitive Engineering and Systems Laboratory. Prior to 
joining Mississippi State, he worked as an instructor of Engineering Science and Mathematics at 
Amarillo College. He received his Ph.D. in Industrial Engineering at Texas Tech University, 
with a specialization in cognitive ergonomics. While completing this degree, he received the Air 
Force Office of Scientific Research Fellowship in Human Factors. He received his Masters and 
Bachelors degrees in Industrial Engineering with specializations in human factors and 
occupational safety from the University of Central Florida. Dr. Kaber's research interests include 
cognitive engineering, adaptive systems and modes of automation in complex systems, 
teleoperations and virtual reality systems. He has published several refereed papers in technical 
journals and conference proceedings concerning these interests. His current projects include 
advanced interface design for teleoperation systems, investigation of the role of telepresence (the 
sense of being present at a remote site) in teleoperation performance, virtual environment design 
for scientific data visualization tasks including meteorological and oceanographic model 
analysis, and assessment of the impact of modes of automation and adaptive automation on 
human operator workload and situation awareness in complex system control tasks. 
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